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A program to solve a wide class of intelligence- 
teat problems of the "geometric-analogy" type ( fig- 
ure A is to figure B as figure C is to which of the 
following figures'!") is being constructed , The pro- 
gram* which is written in LISP., uses heuristic 
methods to (a) calculate j from relatively primitive 
input descriptions, '"articular 9 * (cfo Minaky* Step s 
Toward Artifici al In telligence ) descriptions or the 
figures, then (Bl~ufeili2©lhese descriptions in find- 
ing an appropriate transformation rule and apply- 
ing it, modifying it a3 necessary* to arrive at an 
answer, The current version has solved a number of 
geometric- analogy problems and is now being modified 
in several ways and run on further test cases „ 

The following, intended as a progress report , 
consists of three relatively short sections, con- 
taining (1) a description of the problem type and s 
quits sketohily, the solution process, (2) some 
remarks on the choice of problem and approach* 
and (3) a summary of the steps in the solution of 
a specific sample problem 

I should like to acknowledge the assistance of 
the Cooperative Test Division of BffS and„ in particu- 
lar , Mr c John j» Howell* for very helpfully supply- 
ing an extensive set of geometric- analogy problems 

from th@ir files o 
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Part; I 



The problems to foe considered can be described as follows; 
on© la given a set of eight separate line drawings (each not,, 
in general, connected, and possibly containing dots as well 
as lines)* For later reference* we* 11 say these figures are 
labeled A, B, C* i, 2, 3, 4, and 5, as in the accompanying 
example* The problem is presented to the subject as follows 
(quoted from the tests of the American Council on Education) 5 
"find the rule by which figure A has been changed to make 
figure B Apply the rule to figure Co Select the resulting 
figure from figures 1-5° "' These innocent- sounding instructions 
which people find rather easy to follow (though many of the 
very problems on which the program is being tested were thought 
hard enough to be usefully included on college entrance 
examinations) lead to a number of difficulties when one tries 
to mechanize their execution „ For example* two of these 
difficulties^ together with an indication of the attitudes 
taken toward them in the design of the program, are: 

(a) First, one must develop a suitable way of presenting 
the figures to the machine -> Considerable work has been done 
in various places on methods (using such devices as photocell 
matrices, light-pens, and flying-spot scanners) of going 
from a picture to a representation of it in a form that can 
be handled by computer « Since our basic concern is with 
the lat«*r processing, such problems are bypassed and we start 
off with an Internal representation of the figures in list- 
structure form, read in from punched cards o This representa- 
tion is a quite primitive form of description, however ; it 
could be obtained via any of the input hardware mentioned by 
means no more elaborate than line tracing techniques already 
described in the literature • Roughly, the representation 
permits the description of an arbitrary line drawing to any 
degree of accuracy (but only one "shading" of line is per- 
mitted): a line is represented by a sequence of as many 
straight- line segments and aros of circles as desired* A 
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sample of what this list- structure representation looks Ilka 
is given below, in sec, 3. This representation is convenient 
for our purposes, since it permits quite economical descriptions 
of typical problem figures and lends itself well to programmed 
manipulations. The nature of the required manipulations on 
the figures is indicated later in this section and in sec. 3. 
(b) Next, one must decide what is meant by "rule", i.e„, 
what class of transformations on the figures is to be admitted,, 
Several considerations might govern this choice; (i) one could 
admit only a formally-defined class of rules Ft that, together 
with some formalization of the problem figures, would permit 
one to obtain "theorems* about the resulting system, eg*, that 
a given problem has a unique solution within the permitted 
rule set, i.e*, that there is a unique rule T in R such that 
T(A) - B and T(C) « exactly one of the figures I-55 or (li) 
one could attempt to achieve a very high level of perform- 
ance by gathering a set of essentially ad hoc tricks, well- 
adapted for success on a few particular types of problem of 
interest, and restricting the inputs to these types. However, 
since our object, described in seco 2, is not the formal study 
or maximally efficient solution of geometric-analogy problems 
per se, but the study of the use of descriptive-language methods 
In problem-solving, for which these problems seem to provide 
a fruitful subject matter, our approach is rather different, 
as Will be seen from the description of the process below. 
(Though rejecting the first alternative, we do have formally- 
defined rules that can be manipulated formally to obtain new 
ones; though rejecting the second, we do have formally-defined 
rules that can be manipulated formally to obtain new ones; 
though rejecting the second, we do employ techniques special 
to geometric problems and^ in fact, to restricted classes of 
theai — however, considerable pains have been taken to avoid 
ad hoc solutions which do not contribute to the main goals of 
study) » 

What follows is a brief summary of the entire solution 



process . A problem of the type described Is chosen. The 
corresponding primitive descriptions are written down for 
each of the eight figures and punched on cards • These cards 
are the input to part I of the program (because of storage 
considerations, the current program is segmented into two 
large blocks which occupy core at different times -see sfec« $)» 
The first step is to decompose each figure into "objects". 
The current decomposition program is quite simple-minded; it 
merely divides a figure into its connected parts, e«g„, fig- 
ure A in the example consists of the three objects labeled Pi, 
P2, and P3o Eventually, one would like to have a more sophisti- 
cated decomposition program with, say, the capability of sep- 
arating overlapped objects on appropriate cues, e.g., suppose 
figo A is [^ and f igo B is (Z} . The decomposition program 
should be able to decompose fig- A into the rectangle and 
triangle on the information that these objects are present in 
figo Bo Such a more elaborate decomposition program (which 
has been partly designed) would be of considerable Interest, 
both for itself, as an interesting manipulation on line draw- 
ings, and for the extension in problem- solving power it would 
permit . However, the presence or absence of such a program 
is immaterial to what follows, since the system has been 
written so that such a program may be Installed simply by remov- 
ing the current decomposition program and adding the new one, 
with no other changes any place (except a few in the property 
and relation subroutines (see below), which are regarded as 
variable "parameters * of the part I program) . The "objects™ 
permitted as output of the decomposition routine need not even 

be connected.. 

Next, the objects thus generated are given to a routine 
which calculates a specified set of properties of these objects 
and relations between thera<> As stated above, the program 
designed so that this set can conveniently be changed. Further- 
more, lists of these properties and relations are program 
parameters, specifying which of the ones currently present in 
the system are to be calculated on a given run. The current 
relation-calculation control program permits only two-place 
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relations, but this could be changed without much roubles 
the part II program is written to accept information about 
arbitrary n-place relations* As a sample of a relation- 
calculating subroutine, there is one that calculates in figure A 
of the sample problem* that the object P2 lies in3lde that 
labeled P3 and outputs a corresponding expression* (INSIDE 
F2 P3) The method involves calculating all intersections 
with P3 of a line segf^nt drawn from a point on P2 to the edge 
of the field (all figures are considered as drawn on a unit 
square) . In this case P2 lies inside P3 since the number of 
such intersections is odd, namely i ( and P3 is known to be 
a simple closed curve) . This calculation indicates the 
substantial repertoire of '"analytic geometry" routines required 
for part I, to determine , for example* intersections of 
straight line segments and arcs of circles in all cases and 
combinations o These routines are also heavily used In the 
Similarity calculations, which, aside from the property and 
relation calculations , are the principal business of part I, 
What is calculated, for each appropriate pair of objects is 
a relation which is a slightly extended form of Euclidean simil- 
arity. Namely, th® transformation tested for consists of a 
horizontal, vertical, or null reflection, followed by a Euclid- 
ean similarity transformation (uniform scale change and rota- 
tion), followed again by a reflection, again chosen (independ- 
ently of the first) from horizontal, vertical, or nullo Part I 
contains a set of routines that, given two arbitrary line 
drawings X and Y, will calculate all instances of the above 
transformation taking X into Y (more precisely, making X con- 
gruent to Y up to certain metric tolerances which are para- 
meters in the corresponding programs )« This routine ls^ in 
effect, a pattern recognition program, with built-in invari- 
ance under scale changes, rotations, and certain types of 
reflections o It consists essentially of a topological match- 
ing process, with metric comparisons being made between lines 
the topological matching selects <> Incidentally, it would be 
easy to suppress the metric parts to obtain a program which 



is a completely general topological equivalence test for net- 
works* At any rate* this similarity information is computed 
for every required pair of objects, both within a figure and 
between figures, and this information,, together with the 
property and relation information* is punched out on cards 
in a standard format for input to part Ho (For a typical set 
of figures, the total output of part l f punched at up to 72 
col • So/card, might come to perhaps 45 to 20 cards )<, 

Part II is given these cards as input * Its final out- 
put is either the number of the answer figure or the? state- 
ment that it failed to find an answer (the selective trace 
printing facility of LISP provides an easy and flexible means 
of following the steps in the problem- solving process )o The 
first step generates a rule (or sometimes several alternate 
rules) which transforms figure A into figure Bo Such a rule 
specif ies* as we shall see in a specific case in sec« 3^ how 
the objects of figure A are removed, added to f or altered in 
their properties and relations to other objects to generate 
figure Bo Once this set of rule possibilities has been gen- 
erated* all subsequent effort is devoted to trying to "gen- 
eralize 80 on^ of these rules Just enough so that the resulting 
rule* which still takes figo A into flgr B* now takes figo C 
into exactly one of the answer figures o This requires a 
quite complex mechanism for manipulating and testing ttm 
rules and criteria for deciding which of several rule candld- 
afcfts, the results of different initial rules or of different 
"generalizations 03 * Is to be chosen* The principal method 
embodied in part II at present is able to deal quite generally 
with problems in which the number of parts added * removed * and 
matched in taking figo A into figo B are the same as the num- 
ber of parts added* removed, and matched* respectively in tak- 
ing figo C into the answer figureo A substantial majority of 
the ACE test questions are of this type;, virtually all would 
be under a sufficiently 88 foresighfeed w decomposition process * 
This restriction still permits a wide variety of transformation 
rules o Supplementing this method* which is currently working 
in full generality, other* rather more specialised* methods 
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are being developed to oak* appropriate rule "generalizations" 
to handle other type* of problem* fro* the ftlasa characterized 
above. It should be mentioned that, so far, at least, all the 
methods of part IX have been kept subject-eat tar- free in the 
aenae that no use ia aade of any gaosjetri© properties of the 
properties and relations appearing in the Input to part XI. 
The acre detailed workings of both parts I and II are best 
explained through an exanple; see. 3 io d e voted to a typical 
one* 
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Part; II 

The motivations in choosing geome trie- analogy problems 
as the subject matter for a problem- solving program center 
around the notion of descriptive "languages W o One can argue 
that powerful problem- solving programs must have a good inter- 
nal "linguistic 98 representation of the problems they deal with 
and of the methods they have available for attacking fehese 
problems * In particular* this seems a requirement for programs 
having very much more sophisticated learning capabilities 
than those of present- generation programs * The current pro- 
gram represents a relatively modest exploration and exploita- 
tion of this idea at three levels; (i) firsts the inputs to 
part I constitute a representation of the problem figures in 
a form convenient for the manipulations of that part; (ii) the 
outputs from part I again constitute a representation of the 
figures* in a quite different* more "abstract** form* mora 
convenient for the manipulations of part II5 (ill) finally,, 
the rule expressions generated and altered in part II are 
representations of geometric transformations in a form con- 
venient for the manipulations carried out on them in the 
course of arriving at a solution <, 

I hope that the id^&s embodied in the figure descriptions 
and th§ associated processing techniques mentioned in (i) and 
(ii) above may contribute rather directly to efforts to con- 
struct programs to process line drawings in a variety of 
applications o Furthermore^ the methods associated with the 
rule descriptions of (ill) nay -bo suggestive^ if not directly 
applicable^ In the development of true * theory- forming w prob- 
lem-solving programs * Our part II may be viewed as a rather 
modest example of a program which forms a "theory 88 (the A ** B * 
rule) on the basis of some evidence^ then "generalizes 19 this 
theory p as required, 1 to fit further evidence (figo C) as well, 
then makes a pr<£diction from this theory and tests it on the 
basis of some experimental criterion (here, that the trans- 
formation under consideration gives a unique answer figure). 
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Part III 

To begin the discussion of the sample problem shown in 
the accompanying figure, I»ll give part of the input to part I, 
namely the input description of fig. A. It looks like: 
( (DOT (0.4 . 0o8)) 

(SCO ((0.3 . 0.2) 0.0 (0.7 c 0.2) 0.0 
(0.5 . 0.7) 0.0 (0.3 . 0.2))) 

(SCO ((0.4 . 0.3) 0.0 (0.6 . 0.3) 0.0 (0.6 . 0,4) 
0.0 (0.4 . 0.4) 0,0 (0.4 . 0.3))) ) 
The first line above corresponds to the dot (at coord's. 
x » 0.4 and y « 0.8 on the unit square). The next two lines 
correspond to the triangle (SCC stands for simple closed 
curve: all figures are divided into three classes, dots, 
simple closed curves, and all the rest in the Internal handl- 
ing of their descriptions for reasons of programming conven- 
ience- -no other use is made of this three-way classification) — 
coordinate pairs alternate with the curvatures are zero here 
sine* the lines in question are all straight). Similarly, the 
final two lines correspond to the rectangle? the entire descrip- 
tion is a list of the descriptions of these three parts. The 
format corresponding to non-SCC figures like the Z of fig. C 
is similar though somewhat more complex; the top level des- 
cribes the connectivity by stating the connections between 
the vertices — subllsts describe the lines Joining them 

This input and the corresponding input for the other 
seven figures is processed and the output from part I is, in 
its entirety, as follows (in this example, for brevity in 
writing, since this problem can be handled without them, I 
have omitted all similarity transformation information which 
contains non-null reflections). The output consists of ten 
expressions: (X have replaced the symbols generated inter- 
nally for the parts found by the decomposition program by the 
part names (PI, etc.) which appear on the accompanying figure). 
(!) ((PI P2 P3). ((INSIDE P2 P3) (ABOVE Pi P3) (ABOVE PI P2))) 
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(2) ((F4 P5MCLEFT P4 P5))) 

(3) {(P6 P7 P8}.( (INSIDE P7 P6) (ABOVE P8 P6) (above P8 P7))) 

(4) ((P2 P4 (((1.0 o OoO)o(N.N)) ((1.0 o3oi4). (N.N)))) 

(P3 P5 (((1*0 . 0.0). (N.N))))) 

(5) ((PI P8 (((i.0 . 0.0). (N.N))))) 

(6) NIL 

(7) ( (P9 P10 Pll) (P12 P13) (P14 P15) (P16 P17) (P18) ) 

(8) •( (<h»iab pio -pii) (mm® ?u .?$») (mm vn ?m)) 

({left Fia ?i3)5 Uimim pis ?**)) ({above ri? ?X6)5 

NIL ) 
(9) ( ((P6 P9 (((1«0 . 0.0). (N.N)))) (P7 P10 (((1.0 . e 0), 

(NoN)) ((1.0 o -3-*A)*(NOl))).) (P8 PH (((1.0 . 0.0)« 

(NoN))))) 
((P6 P13 .(((1.0 o OoO)o(NoN)))) (P7 P12 (((1.0 . 0,0), 

(N.N)) ((loO . -3.**).(N.H))).)) 
((P6 P14 (((loO . 0.0). (N.N)))) (P7 P15 (((1.0 . 0.0). (N.N)) 

((1.0 . -3ol4).(N.N))))) 
((P6 P16 (((1.0 o O.O).(N.N)))) (P8 P17 (((loO . 0.0), 

(NoN))))) 
((Ff P18 (((1.0 c 0*0). (N.N))))) ) 

(**) ( ( (( pi pil (((l o0 . 0.0). (NJN))))) NIL NIL 
((PI P17 (((1.0 o O.O)o(NoN))))) NIL ) 
. (NIL NIL NIL NIL NIL) ) 

To explain some of this: the first expression corresponds bo 
fig. A. It ways fig. A has been decomposed into three parts , 
which have been given the names PI, P2, and P3. Then we have 
a list of properties and relations and similarity infromation 
internal to fig, A, namely, here, that P2 is inside P3, Pi is 
above P2, and Pi is above P3. The next two expressions give 
the corresponding information for figs. B and Co The fourth 
expression gives Information about similarities between fig. A 
and fig. B. For example, P3 goes into P5 under a "scale factor 
- 1, rotation angle » 0, and both reflections null" transform- 
ation. The next two expressions contain the corresponding 
information for fig. A to fig. C and from fig. B to fig. C g 
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respectively. The seventh Hat Is a ^element list of lists 
of the parts of the five answer figure*; the eighth a 5-ele- 
ment list of lists, one for each answer figure, giving prop. 
rel., and sim. information., The ninth is again a 5-element ' 
list, each a "similarity" list from fig. c to one of the ans- 
wer figs. The tenth, and last, expression is a dotted pair 
of expressions, the first again a 5-element list, a "similar- 
Ity list from fi go A to each of the answer figures, the sec- 
ond the same from fig. B to each of the answer figures. This 
brief description leaves a lot of loose ends, but it should 
indicate what 9 3 going on. 

Now these ten expressions are given as arguments to the 
top-level function of part II (optimistically called solve). 
The sub-method of solve which suffices to do this problST 
begins by matching the parts of fig. A and those of fig. B 
in all possible ways compatible with the similarity inform- 
ation. Prom this process, it concludes, in the case in 
question, that P2 - P4, p 3 - P 5 , and PI is removed in going 
from A to B. (The machinery can also handle far more compli- 
cated cases, m which alternate matchinga are possible and 
parts are both added and removed). On the basis of this 
matching, a statement of a rule taking a into B Is generated. 
It looks like: 

((REMOVE Ai ((ABOVE Al A3) (ABOVE Al A2) (SIM OB3 Al 
(((1.0 .0.0). (N.N)))))) (MATCH A2 (((INSIDE A2 A3) 
(ABOVE Al A2) (SIM <B2 A2 (((llo . 0.0). (N.N))))) 
((LEFT A2 A3) (SIM GB2 A2 (((1.0 .0.0). (N.N)) 
((1.0.. 3.iU.(N.N)))) (SIMTRAN (((1.0 . 0.0). (N.N)) 
((1.0 . 3-i4),(N.N)))))))(MATCH A3 (((INSIDE A2 A3) 
(ABOVE Al A3) (SIM OBI A3 (((1.0 . 0.0). (N.N))))) 
((LEFT A2 A3) (SIM OBI A3 (((1,0 . 0.0). (N.N)))) 
(SIMTRAN (((1.0 . 0.0). (N.N)))))))) 

The A's are used as "variables" representing objects. The for- 
mat is rather simple. For each object added, removed, or 
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matched, there is a list of the properties, relations, and 
similarity information pertaining to it. (in the case of a 
matched object, there are two such lists, one pertaining to 
fig. A and the other to fig. B)» There are two special 
devices s the (SIM OBI, o<, -form expressions give a means 
of comparing types of objects between, say, fig, A and fig, Cs 
the other device is the use of the SXHTRAN expressions in the 
fig. B-liat; ror each matched object. This enables us to 
handle conveniently some additional situations that X won't 
attempt to describe here.. 

This rule contains everything about figs A and B and their 
relationship that is used in the rest of the process . (The 
reader may easily verify that the rule does, in some sense,* 
describe the transformation of fig* A into fig. B in the 



Now a similarity matching is carried out between C and 
each of the five answer figures* Mafechinga which don*t 
correspond to the ones between A and B in number of parts 
added, removed, and matched are discarded* Xf all are rejected 
this method has failed end we go on to try some other method D 
In our case, figSe 1 and 5 are rejected on this basis* How- 
ever figao 2, 3 P and 4 pass this test and are examined further, 
as follows: for a given matching of fig» C to the answer fig- 
ure in question (and we will go through all possible matchings 
compatible with similarity) we take each A -*B rule and attempt 
to fit it to the new case, making all matchings of objects 
between the A*s of the rule statement and the objects of C and 
the answer fig* compatible with preserving add, remove, and 
match categories, then testing to see which information is 
preserved, thus getting a new, "reduced" rule which fits both 
A ""* B and C H> the answer figure in question* In our case, for 
each of the three possible answer figures we get two reduced 
rules in this way (since there are two possible pairings 
between A and C, namely, PI ** P8, 5 P2 ** P7, and P3 ** P6, or 
PI ** P8, P2 •* P6, and P3 ** P7) * In some sense, each of these 
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rules provides an answer, However, we want a "best" rule-- 
If one interprets this to mean the "strongest* rule, i o e , 
the one that says the most or is the least alteration in the 
original A -* B rule that fits C -» some answer figure, then 
a simple device seems to approximate human opinion on this 
question rather wells we define a rather simple "strength" 
function on the rules and sort them by this. If a rule is 
a clear winner in this test, the corresponding answer figure 
is chosen? if it results in a tie, the method has failed. In 
our case when the values for the six rules are computed, the 
winner is one of the rules corresponding to figure 2, so 
the program like all humans consulted so far, chooses it as 
the answer* The rule looks like this* 
((REMOVE Al ((ABOVE Al A3) (ABOVE Al A2) (SIM QB3 A3. 
(((1,0 . 0.0) e (N;N)))))) (MATCH A2 (((INSIDE A2 A3) 
(ABOVE Ai A2)).C(LEFT A2 A3) (SIMTRAN (((1.0 . 0.0). (M.X)) 
((1.0 . 3ol4)4N.N))))))) (MATCH A3 (((INSIDE A2 A3) 
(ABOVE Al A3))* ((LEFT A2 A3) (SIMTRAN (((ioO . 0<>0)o 
(NoN)))))))) 

Again, it is easy to check that this rule both takes A into 
B and C into 2, but not into any of the other answer figures „ 

Note ; 

The possibility frequently exists in LISP of writing the 
same S-expresslon in alternate forms, by using either dot or 
list notation, in various combinations. In writing the S- 
sxpressions contained in this memo, I have taken the liberty 
in several places of using this possibility to express them, 
for greater clarity, in a form different from that produced 
by the LISP print program* In all these cases, the meaning 
of the expression in terms of list structure is unchanged by 
this rdwri tinge 



CS-TR Scanning Project ,, c . 

Document Control Form Date JjJJOJ^- 

Report #_M!kJiL. 

Each of the following should be identified by a checkmark: 
Originating Department: 

"^Artificial Intellegence Laboratory (Al) 

□ Laboratory for Computer Science (LCS) 

Document Type: 

□ Technical Report (TR) ^^ Technical Memo (TM) 

□ Other: \ 

Document Information Number of pages: -ISi&jp*?*] 

- Not to include DOD forms, printer intstructions. etc... original pages only. 

Originals are: intended to be printed as : 

p^ Single-sided or ^ Single-sided or 

□ Double-sided D Double-sided 

Print type: 

□ Typewriter □ Offset Press Q Laser Print 

□ InkJet Printer Q Unknown ft Other: ^■h^G^^ti 

Check each if included with document: 

D DOD Form D Funding Agent Form D Cover Page 

D Spine D Printers Notes □ Photo negatives 

□ Other: 

Page Data: 



Blank Pagestbyp^. «««*«): , 

Photographs/Tonal Material <*w ****>'■, 



Other (note Onavlxrivv number^ 

Description : Page Number. 

■x/nKtff mtfi ft - lO TrcuL ?«££ 1 <***• F;cr.3-y, 

**,- H . 

fib- H ) S^r^otJTi^L TfiaTJ (3~^> 



'1- 



Scanning Agent Signoff: , c 

Date Received: JJ_lJl.l5L Date Scanned: JJL/JU3L Date ^turned: J2JJ1JJ~. 

Scanning Agent Signature: /M^Cf^uA 1/ » M?ir£L> RevWD s/Lc^r>xam^ control Form cstffofm.v«i 



Scanning Agent Identification Target 



Scanning of this document was supported in part by 
the Corporation for National Research Initiatives, 
using funds from the Advanced Research Projects 
Agency of the United states Government under 
Grant: MDA972-92-J1029. 



The scanning agent for this project was the 
Document Services department of the M.I.T 
Libraries. Technical support for this project was 
also provided by the M.I.T. Laboratory for 
Computer Sciences. 



Scanned 



M.I.T. Libraries 
Document Services 



darptrgt.wpw Rev. 9/94 



